如何从 PDF 的图表和表格中提取真正的价值

文章背景与核心概要

传统的 RAG（检索增强生成）系统在处理图文混排的 PDF 时往往力不从心，因为它们严重依赖文本提取和光学字符识别（OCR），这会漏掉柱状图、对比图表、架构图和视觉布局中锁定的所有核心信息。

本文探讨了一种现代替代方案：晚期交互多向量检索（Late-Interaction Multi-Vector Retrieval）。通过将整个 PDF 页面视为视觉图像，并将其编码为一组多向量（借助 ColPali/ColBERT 以及 Weaviate 的 multi2multivec-weaviate 等工具），你可以完全跳过 OCR 和文本分块（chunking）。这使得系统能够通过自然语言查询精准检索出准确的图表，利用查询智能体（Query Agent）赋能高级的多步推理，并将生产环境中的数据管道简化为干净、易于管理的代码。

摘要

Traditional RAG (Retrieval-Augmented Generation) pipelines struggle with rich PDFs because they rely on text extraction and OCR (Optical Character Recognition), which completely misses information locked inside bar charts, comparison tables, architectural diagrams, and visual layouts.

This article explores a modern alternative: Late-Interaction Multi-Vector Retrieval. By treating entire PDF pages as visual images and encoding them as a set of multi-vectors (via tools like ColPali/ColBERT and Weaviate's multi2multivec-weaviate), you can completely skip OCR and chunking. This enables systems to precisely retrieve exact charts and tables from natural language queries, power advanced multi-step reasoning via query agents, and simplify production pipelines into clean, manageable code.

简介

If you have ever tried to put a stack of investor decks, scientific papers, or annual reports through a RAG pipeline, then you know the drill: set up an Optical Character Recognition (OCR) or text-extraction step, pick a chunking strategy, embed text, and finally retrieve information. After going through all this, someone may ask a question about revenue in Q2 FY25, and your retrieval would return three pages of unrelated bullet points because the actual answer is in a bar chart that was invisible to your index.

This used to be what people had to do — just leave out the interesting parts of a PDF from RAG such as bar charts with trends, comparison tables, architectural diagrams, and things in general that make PDFs so much more valuable than mere text.

In this article, we'll show you a better way that not only retrieves the rich information embedded in charts, but also eliminates complex processing steps. We'll cover:

• Classic RAG: Why OCR and text embedding works great for some (structured and unstructured) data, but not for rich PDFs.
• Late Interaction RAG: What late-interaction multi-vector models are, and why they let you skip text extraction entirely.
• Drag-N-Drop and Done: How to ingest multiple PDFs in Weaviate Cloud with just a few clicks.
• Real Examples: Example queries against NVIDIA's FY2026 quarterly earnings, where each top result is the exact chart that answers a question.
• Complex Agentic Reasoning: How to wrap the same data with the Weaviate Query Agent for synthesised answers with page-image citations.
• Codified and Deployment-Ready: The same ingestion pipeline in roughly 50 lines of Python for when you need to deploy this in production.

If you'd rather just see the demo first and read the explanations later, skip to the queries section below.

如果你曾尝试将一堆投资者路演幻灯片、科学论文或年度报告放入 RAG 管道中，你就会对这个流程了如指掌：设置 OCR（光学字符识别）或文本提取步骤、选择分块策略、对文本进行嵌入（Embedding），最后进行检索。经历这一切之后，当有人询问关于 2025 财年第二季度 营收的问题时，你的检索结果却返回了三页毫不相干的要点列表，因为真正的答案藏在一个对你的索引而言形同隐形的柱状图里。

过去人们不得不这样做——在 RAG 中直接忽略 PDF 中最有趣的部分，例如带有趋势的柱状图、对比表格、架构图，以及那些让 PDF 的价值远超纯文本的各种视觉元素。

在本文中，我们将为你展示一种更好的方法，它不仅能检索嵌入在图表中的丰富信息，还能省去复杂的处理步骤。我们将涵盖以下内容：

• 传统 RAG：为什么 OCR 和文本嵌入适用于某些（结构化和非结构化）数据，却不适合内容丰富的 PDF。
• 晚期交互 RAG：什么是晚期交互多向量模型，以及为什么它们能让你完全跳过文本提取。
• 拖放即完成：如何在 Weaviate Cloud 中通过几次点击就导入多个 PDF。
• 真实案例：针对英伟达 2026 财年季度财报的查询示例，其中每个顶部结果都是能够回答问题的精确图表。
• 复杂的智能体推理：如何使用 Weaviate 查询智能体包裹相同的数据，以获得带页面图像引用的综合答案。
• 代码化且随时可部署：在生产环境中部署时，仅需大约 50 行 Python 代码即可实现的相同导入管道。

如果你想先看演示再阅读解释，请直接跳到下方的查询部分。

传统 RAG 的不足之处

Let's first look at what most people do today when it comes to building a RAG pipeline for PDFs:

1. OCR (or text-extract) the file (using either a python library or third-party tooling).
2. Chunk the text.
3. Embed the chunks with a text embedding model.
4. Retrieve, (maybe) rerank, generate.

There's nothing inherently bad about this workflow, but there's a naive underlying assumption that the page can be reduced to a sequence of text tokens without losing the richness of the content. For a press release or a Wikipedia article, this is mostly fine, but for a slide deck full of charts, a 10-K with comparison tables, or an academic paper with illustrative figures, it's simply not enough.

You could build a complex ETL pipeline to extract charts and vectorize them separately, but that means another model in the pipeline and introduces more complexity into the stack as well as merging problems at query time.

With a late-interaction multi-vector model, you don't need any of those: You embed the page, and not the text within it. The model sees the chart the way you do. Yes, you heard that right, there is not even a chunking step — the page is the chunk.

让我们首先看看目前大多数人在构建 PDF 的 RAG 管道时所做的事情：

1. 对文件进行 OCR（或文本提取）（使用 Python 库或第三方工具）。
2. 对文本进行分块（Chunk）。
3. 使用文本嵌入模型对文本块进行向量化。
4. 检索、（可能的话）重排、生成。

这个工作流程本身并没有什么不好，但它背后有一个天真的假设：页面可以被简化为一串文本 Token，而不会丢失内容的信息量。对于新闻稿或维基百科文章来说，这通常没问题；但对于充满图表的幻灯片演示文稿、包含对比表格的 10-K 财报，或带有说明性插图的学术论文，这远远不够。

你可以构建一个复杂的 ETL 管道来提取图表并单独进行向量化，但这意味管道中引入了另一个模型，从而增加了技术栈的复杂性，并带来了查询时的结果合并问题。

有了晚期交互多向量模型，你就不再需要这些了：你嵌入的是整个页面，而不是其中的文本。模型像你一样看待图表。是的，你没听错，甚至连分块步骤都没有了——页面本身就是那个块（Chunk）。

晚期交互多向量检索（用两段话概括）

Before we get to the demo, let's take a quick detour on what makes this work. If you already know your way around ColPali / ColBERT, feel free to skip or skim.

Traditional dense embedding models compress an entire document (or chunk) into a single vector. Late-interaction multi-vector models do something different: they encode the document as a set of vectors, typically one per token (or for a vision model like the one we're about to use, one per image patch). At query time, your query is also encoded as a set, and the relevance score is the sum of best matches between query tokens and document tokens (a quantity called MaxSim).

Instead of asking "is this whole page about my whole question?", the model can ask "is the part of this page that talks about Q4 FY26 a good match for the part of my question about 'change over time'?". For a chart-heavy page, this is exactly the granularity that's needed. The model doesn't have to summarise an entire slide into a single vector, but can keep one vector per region of the page, and the query can pick out the regions that matter.

So really, there are two distinct advantages here:

1. Because it's a visual model you keep the layout, the charts, the tables, etc. and
2. By using MaxSim over a set of vectors, you eliminate the need for chunking.

Weaviate offers multi2multivec-weaviate, a vectorizer module that runs a hosted late-interaction multi-vector model on Weaviate Cloud, so you don't have to host or manage a model in order to generate multi-vectors from your PDFs.

在进入演示之前，让我们简单了解一下其背后的工作原理。如果你已经对 ColPali / ColBERT 有所了解，可以跳过或略读此节。

传统的密集嵌入模型（Dense Embedding Models）将整个文档（或文本块）压缩为单个向量。而晚期交互多向量模型则有所不同：它们将文档编码为一组向量，通常每个 Token 对应一个向量（对于我们即将使用的视觉模型来说，则是每个图像补丁 Image Patch 对应一个向量）。在查询时，你的查询也被编码为一个向量集，相关性得分则是查询 Token 与文档 Token 之间最佳匹配的总和（这个量被称为 MaxSim）。

模型不再去问“这一整页是否都在讲我的整个问题？”，而是去问“页面中讨论 2026 财年第四季度的部分，是否与我问题中关于‘随时间变化’的部分高度匹配？”。对于图表密集的页面，这正是所需的粒度。模型无需将整张幻灯片总结为一个向量，而是可以为页面的每个区域保留一个向量，查询则可以挑选出那些重要的区域。

因此，这里实际上有两个明显的优势：

1. 因为它是一个视觉模型，你得以保留了布局、图表、表格等所有元素；并且
2. 通过在一组向量上使用 MaxSim，你彻底省去了分块（Chunking）的需要。

Weaviate 提供了 multi2multivec-weaviate，这是一个在 Weaviate Cloud 上运行托管的晚期交互多向量模型的向量化器模块，因此你无需自行托管或管理模型即可从 PDF 生成多向量。

通过拖放导入 PDF

The fastest way to try this is the drag-and-drop importer in Weaviate Cloud.

1. Open your cluster in the Console.
2. Go to Collections and create a new collection.
3. Select the upload from file option and drop in your PDFs.

And that's the entire ingestion process. Behind the scenes, Weaviate is doing three things:

1. Rendering each PDF page to a high-resolution image.
2. Storing that image as a BLOB property on a new collection.
3. Vectorizing it with the multi2multivec-weaviate module, producing many vectors per page.

A few things are worth noting:

• One object equals one page. The unit of retrieval is the page, which is also the unit a human navigates a document by.
• No OCR. The model never sees the text as text. It sees the page as an image. That's why a chart with no caption is just as searchable as a paragraph.
• The vectors are compressed. Late-interaction models can produce hundreds of vectors per page, which would be expensive to store naively. Weaviate uses a multi-vector encoding scheme that keeps the index compact. (More on that also in the trade-offs section below.)

For this demo, we've imported NVIDIA's four FY2026 quarterly investor presentations (Q1 through Q4). They cover the financial year ending in January 2026, contain wall-to-wall charts and tables, and total 92 pages. The whole import took about a minute and a half.

体验这一功能的捷径是使用 Weaviate Cloud 中的拖放导入工具。

1. 在控制台中打开你的集群。
2. 转到 Collections 并创建一个新的集合（Collection）。
3. 选择从文件上传选项，并将你的 PDF 拖入其中。

整个导入过程就是这样。在幕后，Weaviate 正在做三件事：

1. 将每个 PDF 页面渲染为高分辨率图像。
2. 将该图像作为 BLOB 属性存储在新集合中。
3. 使用 multi2multivec-weaviate 模块对其进行向量化，为每个页面生成多个向量。

有几点值得注意：

• 一个对象等于一个页面。 检索的单位是页面，这也是人类浏览文档时的单位。
• 没有 OCR。 模型从不将文本视为文本。它将页面视为图像。这就是为什么没有标题的图表和段落一样易于搜索的原因。
• 向量经过了压缩。 晚期交互模型每个页面可以产生数百个向量，如果直接存储，成本会很高。Weaviate 使用了多向量编码方案，使索引保持紧凑（关于这一点，我们将在下文的“权衡”部分详细介绍）。

在此次演示中，我们导入了英伟达 2026 财年的四个季度投资者演示文稿（第一季度到第四季度）。它们涵盖了截至 2026 年 1 月结束的财年，里面全是图表和表格，共计 92 页。整个导入过程大约只花了分半钟。

查询英伟达的季度财报

Before we do anything fancy such as agentic reasoning on the data, we want to show you the raw retrieval results, as they are already impressive on their own.

With ingestion complete, you can query the data directly in the Console (or with any Weaviate client). The query is in plain English, and the result is a ranked list of pages with page images inline.

Let's walk through three queries and show you what results are returned.

在我们对数据进行诸如智能体推理之类的花哨操作之前，我们想向你展示原始的检索结果，因为它们本身就已经足够令人惊叹了。

导入完成后，你可以直接在控制台中（或使用任何 Weaviate 客户端）查询数据。查询使用的是日常英语，结果则是按相关性排名的页面列表，并内联了页面图像。

让我们通过三个查询来看看返回了什么结果。

查询 1：“汽车业务营收是如何随时间变化的？” (“how did automotive revenue change over time?”)

The top result is a single page from the Q2 FY26 deck, titled Automotive. The left half is a bar chart showing five quarters of revenue ($346M → $449M → $570M → $567M → $586M, +69% Y/Y). The right half contains three bullet points about Thor SoC and DRIVE AV.

The phrase "over time" doesn't appear anywhere on this page. Neither does the word "change". The model didn't match against text semantically, but rather the image of the page. What it saw was five bars of increasing height with quarterly labels, and that was enough to identify the page as a match for a question about a temporal trend.

排名字首位的结果是 Q2 FY26 演示文稿中的单页，标题为 Automotive（汽车业务）。左半部分是一个柱状图，展示了五个季度的营收（$346M → $449M → $570M → $567M → $586M，同比增长 69%）。右半部分包含关于 Thor SoC 和 DRIVE AV 的三个要点。

短语“over time”（随时间推移）并没有出现在这页的任何地方，单词“change”（变化）也没有。模型并不是通过文本进行语义匹配的，而是通过页面的图像。它看到的是五个高度不断增加、带有季度标签的柱子，这就足以将该页面识别为与时间趋势相关问题的匹配项。

查询 2：“毛利率趋势” (“gross margin trend”)

The top result here is the Q2 FY26 Financial Summary page. On the left is a combination chart: revenue bars and a non-GAAP gross margin line over five quarters. On the right is a GAAP/non-GAAP KPI table with Y/Y and Q/Q deltas.

Again, nothing on this page literally says "gross margin trend", but there is a line chart that visualises the gross margin dipping from 75.7% to 61.0% in Q1 FY26 and recovering to 72.7% in Q2 FY26. That's what a trend looks like, and it's what the model retrieved.

A side note: the same page also contains a detailed financial table. That makes it a particularly useful retrieval target if you're then going to ask follow-up questions like "by how many basis points did gross margin recover Q/Q?" The answer is sitting on the page the model already returned. More on that below when we introduce the Query Agent.

这里的首要结果是 Q2 FY26 财务摘要 页面。左侧是一个组合图表：五个季度的营收柱状图和非美国通用会计准则（Non-GAAP）毛利率折线图。右侧是一个包含同比和环比变动的 GAAP/Non-GAAP KPI 表格。

同样，这一页上没有任何文字字面提到“毛利率趋势”，但确实有一个折线图直观地展示了毛利率在 Q1 FY26 从 75.7% 跌至 61.0%，并在 Q2 FY26 恢复到 72.7% 的过程。这就是趋势的样子，也是模型所检索到的内容。

顺便提一句：同一页还包含一个详细的财务表格。如果你随后要提出诸如“毛利率环比回升了多少个基点？”之类的后续问题，这会使它成为一个特别有用的检索目标。答案就躺在模型已经返回的页面上。我们在下面介绍查询智能体时会详细说明这一点。

查询 3：“数据中心营收是如何随时间变化的？” (“how did data center revenue change over time?”)

The top result is the Q4 FY26 Revenue page — a Y/Y bar chart ($39.3B → $68.1B) with a callout that data center revenue is up 13x since the emergence of ChatGPT. This is a great example that the model still respects text when it's the better match. In this case the chart is less relevant, but the box stating the exact answer is what returned the highest similarity (MaxSim) on this page. So you get the best of both modalities.

The runner-up is the dedicated Data Center page from a different quarter, which splits the segment into Compute and Networking:

Notice how the second-best match isn't simply "another page about data center", but a different kind of answer — the same revenue, broken down differently. That's a useful property for an agent that wants to triangulate across multiple views of the same underlying number. Speaking of which...

首要结果是 Q4 FY26 的营收页面——一个同比柱状图（$39.3B → $68.1B），并附带了一个标注：自 ChatGPT 问世以来，数据中心营收增长了 13 倍。这是一个极好的例子，表明当文本是更好的匹配项时，模型依然尊重文本。在这种情况下，图表的重要性相对较低，但包含确切答案的文本框在该页面上返回了最高的相似度（MaxSim）。因此，你同时获得了两种模态的优点。

次优结果来自另一个季度的专用数据中心页面，该页面将该细分领域拆分为计算（Compute）和网络（Networking）：

请注意，第二好的匹配项并非简简单单的“另一页关于数据中心的页面”，而是另一种类型的答案——同样的营收，但进行了不同的拆解。对于想要针对同一底层数字从多个视角进行交叉验证的智能体来说，这是一个非常有用的特性。说到这里……

从搜索到答案：Weaviate 查询智能体

Vector search returns results, but sometimes you want an answer.

The Weaviate Query Agent is a managed agent that wraps vector retrieval with multi-step reasoning, source citations, and inline page images. It is available out of the box for any Weaviate Cloud cluster. Simply point it to a collection and ask a question.

If we ask "How did automotive revenue change across FY26 quarters? What's driving it?" about the same collection, the agent comes back with a synthesised answer (the numbers from the bar chart, plus the bullet points about Thor SoC and DRIVE AV adoption) and the underlying page images as citations. The agent decided which pages to retrieve, looked at them visually, and quoted directly from the slide.

Open the Sources panel and you'll see exactly which pages backed the answer. Among the citations is the Automotive page from the Q1 FY26 deck — the same kind of bar chart we surfaced in the raw vector search earlier, just for a different quarter.

Every numerical claim in the response is anchored to a specific page in a specific PDF, with the page image right there for verification. You don't have to blindly trust the agent, but can read the contents of the slide for yourself.

向量搜索返回的是结果，但有时你想要的是答案。

Weaviate 查询智能体（Query Agent）是一个托管智能体，它将向量检索与多步推理、来源引用和内联页面图像结合在了一起。任何 Weaviate Cloud 集群都可以开箱即用。只需将它指向一个集合并提出问题即可。

如果我们针对同一个集合提出问题：“汽车营收在 2026 财年各季度是如何变化的？驱动因素是什么？”，智能体将返回一个综合答案（柱状图中的数字，加上关于 Thor SoC 和 DRIVE AV 采用情况的要点），并以底层页面图像作为引用。智能体决定检索哪些页面，视觉上“观察”它们，并直接引用幻灯片中的内容。

打开来源（Sources）面板，你就会看到究竟是哪些页面支撑了这个答案。引用的内容中包括来自 Q1 FY26 演示文稿的 Automotive 页面——也就是我们刚才在原始向量搜索中浮现出的同类柱状图，只是换了一个不同的季度。

响应中的每一个数字声明都锚定在特定 PDF 的特定页面上，并且页面图像就在那里供验证。你不必盲目信任智能体，而是可以亲自阅读幻灯片的内容。

用 Python 构建可部署的管道

The drag-and-drop UI is the fastest path to creating a POC, but most production pipelines need code, and here is the equivalent in roughly 50 lines.

First, install the dependencies:

pip install "weaviate-client>=4.21" PyMuPDF

Then, render each PDF page to a 2000-pixel PNG and store it as a BLOB in a collection vectorized with multi2multivec-weaviate:

import os
from base64 import b64encode
from pathlib import Path

import fitz  # PyMuPDF
from weaviate import connect_to_weaviate_cloud
from weaviate.classes.config import Configure, DataType, Property


def page_to_b64(page, long_edge: int = 2000) -> str:
    scale = long_edge / max(page.rect.width, page.rect.height)
    pix = page.get_pixmap(matrix=fitz.Matrix(scale, scale))
    return b64encode(pix.tobytes(output="png")).decode()


client = connect_to_weaviate_cloud(
    os.environ["WEAVIATE_URL"],
    auth_credentials=os.environ["WEAVIATE_API_KEY"],
)

if not client.collections.exists("PDF"):
    client.collections.create(
        name="PDF",
        properties=[
            Property(name="pdf_name", data_type=DataType.TEXT),
            Property(name="page_number", data_type=DataType.INT),
            Property(name="page_image", data_type=DataType.BLOB),
        ],
        vector_config=Configure.MultiVectors.multi2vec_weaviate(
            image_field="page_image",
        ),
    )

col = client.collections.get("PDF")
for pdf_path in Path("pdfs").glob("*.pdf"):
    with col.batch.fixed_size(batch_size=2) as batch, fitz.open(pdf_path) as doc:
        for i, page in enumerate(doc, start=1):
            batch.add_object(properties={
                "pdf_name": pdf_path.name,
                "page_number": i,
                "page_image": page_to_b64(page),
            })

client.close()

A few notes on the above:

• The Python call MultiVectors.multi2vec_weaviate(image_field="page_image") configures the multi2multivec-weaviate module. This is the same vectorizer used by the Console import UI.
• PyMuPDF does the rasterization. The 2000 pixel long-edge target is a sensible default; smaller targets saves time while larger gives the model more detail. We haven't found a strong case for going below 1500 or above 2500.
• batch_size=2 is intentional. Each object carries a multi-megabyte image, so small batches keep the gRPC payload sane.

Query the created collection to return ranked pages:

from weaviate.classes.query import MetadataQuery

res = col.query.near_text(
    query="how did automotive revenue change over time",
    limit=5,
    return_properties=["pdf_name", "page_number"],
    return_metadata=MetadataQuery(distance=True),
)
for o in res.objects:
    print(o.metadata.distance, o.properties)

On the NVIDIA corpus, this returns the Q2 FY26 Automotive page as result #1: the same five-quarter bar chart you saw above, retrieved by an end-to-end pipeline that contains zero OCR.

The same Query Agent you saw in the Console is also available from Python. Pass it the collections to reason over, then ask():

from weaviate.agents.query import QueryAgent
from weaviate_agents.classes import QueryAgentCollectionConfig

agent = QueryAgent(
    client=client,
    collections=[
        QueryAgentCollectionConfig(name="PDF"),
    ],
)

response = agent.ask("How did automotive revenue change across FY26 quarters? What's driving it?")
response.display()

response.display() renders the same synthesised answer plus page-image citations you saw in the Console.

拖放界面是创建 POC（概念验证）的最快途径，但大多数生产管道都需要代码，以下是用大约 50 行代码实现的等效方案。

首先，安装依赖项：

pip install "weaviate-client>=4.21" PyMuPDF

然后，将每个 PDF 页面渲染为 2000 像素的 PNG，并将其作为 BLOB 存储在使用 multi2multivec-weaviate 向量化的集合中：

import os
from base64 import b64encode
from pathlib import Path

import fitz  # PyMuPDF
from weaviate import connect_to_weaviate_cloud
from weaviate.classes.config import Configure, DataType, Property


def page_to_b64(page, long_edge: int = 2000) -> str:
    scale = long_edge / max(page.rect.width, page.rect.height)
    pix = page.get_pixmap(matrix=fitz.Matrix(scale, scale))
    return b64encode(pix.tobytes(output="png")).decode()


client = connect_to_weaviate_cloud(
    os.environ["WEAVIATE_URL"],
    auth_credentials=os.environ["WEAVIATE_API_KEY"],
)

if not client.collections.exists("PDF"):
    client.collections.create(
        name="PDF",
        properties=[
            Property(name="pdf_name", data_type=DataType.TEXT),
            Property(name="page_number", data_type=DataType.INT),
            Property(name="page_image", data_type=DataType.BLOB),
        ],
        vector_config=Configure.MultiVectors.multi2vec_weaviate(
            image_field="page_image",
        ),
    )

col = client.collections.get("PDF")
for pdf_path in Path("pdfs").glob("*.pdf"):
    with col.batch.fixed_size(batch_size=2) as batch, fitz.open(pdf_path) as doc:
        for i, page in enumerate(doc, start=1):
            batch.add_object(properties={
                "pdf_name": pdf_path.name,
                "page_number": i,
                "page_image": page_to_b64(page),
            })

client.close()

关于上述代码的几点说明：

• Python 调用 MultiVectors.multi2vec_weaviate(image_field="page_image") 配置了 multi2multivec-weaviate 模块。 这与控制台导入 UI 使用的是同一个向量化器。
• PyMuPDF 负责进行栅格化（Rasterization）。 将长边目标设为 2000 像素是一个合理的默认值；较小的目标可以节省时间，而较大的目标则能为模型提供更多细节。我们发现低于 1500 或高于 2500 并没有太大必要。
• batch_size=2 是有意为之的。 每个对象都承载着一个多兆字节的图像，因此小批次可以保持 gRPC 负载在合理范围内。

查询创建的集合以返回排好序的页面：

from weaviate.classes.query import MetadataQuery

res = col.query.near_text(
    query="how did automotive revenue change over time",
    limit=5,
    return_properties=["pdf_name", "page_number"],
    return_metadata=MetadataQuery(distance=True),
)
for o in res.objects:
    print(o.metadata.distance, o.properties)

在英伟达语料库上，这将返回 Q2 FY26 的 Automotive 页面作为结果 #1：即你在上面看到的同一个五季度柱状图，它是由一个完全不包含 OCR 的端到端管道检索出来的。

你在控制台中看到的同一个查询智能体（Query Agent）也可以在 Python 中使用。将要推理的集合传给它，然后调用 ask()：

from weaviate.agents.query import QueryAgent
from weaviate_agents.classes import QueryAgentCollectionConfig

agent = QueryAgent(
    client=client,
    collections=[
        QueryAgentCollectionConfig(name="PDF"),
    ],
)

response = agent.ask("How did automotive revenue change across FY26 quarters? What's driving it?")
response.display()

response.display() 将渲染出你在控制台中看到的相同综合答案以及页面图像引用。

那么，这是处理 PDF 的银弹吗？

Of course not. As always in Engineering, there are trade-offs:

• Many vectors per object/page. Late-interaction multi-vector models produce many vectors per page. You can partially offset this with compression techniques, such as Muvera, which are natively supported in Weaviate. This helps, but also introduces a compression/accuracy trade-off. For a corpus that is dominated by large amounts of plain text (think legal contracts, transcripts, or log files), a text embedding model will still be cheaper and just as accurate.
• Page-level retrieval is coarse. If your answer lives in one paragraph buried in a dense contract, returning a whole page may be more context than you want. In practice, we see this typically mitigated in the agent layer (e.g. Weaviate's Query agent), where the relevant paragraph can be identified without requiring a significant number of tokens.
• The model has to understand your charts. It generalises well from the public corpora it was trained on, but if you have very domain-specific visual conventions (think highly stylised internal templates), you should validate retrieval quality on it first.

So when does this approach with late-interaction multi-vector models win? Mostly in PDF-heavy domains that contain assets such as quarterly slide decks, due-diligence packets, scientific figures, and technical drawings, etc. For text-dominated corpora, a text pipeline is most likely cheaper and sufficiently performant.

For further optimization, you might consider using a hybrid approach, where you identify pages with charts to implement as multi-vectors and the remaining corpus as text, and then use an RRF-style approach to merge results at query time.

当然不是。正如工程学中常有的情况一样，这里也存在权衡：

• 每个对象/页面有多个向量。 晚期交互多向量模型每个页面会产生多个向量。你可以通过压缩技术（例如 Weaviate 原生支持的 Muvera）来部分抵消这一点。这有所帮助，但也会引入压缩与准确率之间的权衡。对于以大量纯文本为主的语料库（例如法律合同、会议记录或日志文件），文本嵌入模型依然更便宜且同样准确。
• 页面级检索较粗糙。 如果你的答案隐藏在密密麻麻合同中的某一个段落里，返回整页可能会带来超出你期望的上下文。在实践中，我们发现这通常可以在智能体层（例如 Weaviate 的查询智能体）得到缓解，在这一层中，无需消耗大量 Token 即可识别出相关的段落。
• 模型必须理解你的图表。 它能很好地从其接受训练的公共语料库中进行泛化，但如果你有非常特定于领域的视觉惯例（例如高度风格化的内部模板），你应该先验证其检索质量。

那么，采用晚期交互多向量模型的这种方法何时最有效呢？主要是在 PDF 密集型的领域，其中包含诸如季度幻灯片演示文稿、尽职调查包、科学图表和技术图纸等资产。对于以文本为主的语料库，文本管道很可能更便宜且性能足够。

为了进一步优化，你可以考虑使用混合方法，即识别出包含图表的页面并将其实现为多向量，而将其余语料库作为文本，然后在查询时使用 RRF（倒数排序融合）风格的方法来合并结果。

结论

Let's wrap up with the key takeaways:

• Charts and tables are not problems to be solved by better OCR. They are primary information that gets destroyed in extraction. If your pipeline only indexes text, valuable information from charts is already lost before you started embedding.
• Late-interaction multi-vector models let you skip extraction entirely. Render the page, embed the image, and ask questions about what the image means. The model retrieves pages by what they look like --- charts, tables, layout, and all.
• On Weaviate Cloud, ingestion is drag-and-drop. Point the Query Agent at the resulting collection, ask a question, and get an answer with proper images as citation.
• Turn the same primitive into deployable code. MultiVectors.multi2vec_weaviate(image_field="page_image") is the entire vectorizer config. The rest is rasterising pages with PyMuPDF and a small batch.add_object loop.

If you have been working around chart-heavy PDFs because the indexing pipeline made them too painful to deal with, this is worth a try. The kind of files you used to skip, because they were "mostly charts", are exactly what this approach is built for.

Spin up a Weaviate Cloud cluster, upload your PDFs, and start asking questions today.

让我们用核心要点来做个总结：

• 图表和表格不是靠更好的 OCR 就能解决的问题。 它们是核心信息，却在提取过程中被破坏了。如果你的管道只索引文本，那么图表中宝贵的信息在开始嵌入之前就已经丢失了。
• 晚期交互多向量模型让你完全跳过提取步骤。 渲染页面、嵌入图像，然后询问图像代表什么意思。模型通过页面的外观来检索页面——图表、表格和布局一应俱全。
• 在 Weaviate Cloud 上，导入只需拖放。 将查询智能体指向生成的集合，提出问题，即可获得带有恰当图像作为引用的答案。
• 将相同的原语转化为可部署的代码。 MultiVectors.multi2vec_weaviate(image_field="page_image") 就是全部的向量化器配置。剩下的就是用 PyMuPDF 栅格化页面以及一个小的 batch.add_object 循环。

如果你一直因为索引管道难以处理而对图表密集的 PDF 感到头疼，那么这个方法值得一试。那些你过去因为它们“全都是图表”而选择跳过的文件，正是这种方法为之而生的对象。

立即启动一个 Weaviate Cloud 集群，上传你的 PDF，开始提问吧。

准备好开始构建了吗？

Check out the Quickstart tutorial, or sign up for a free Weaviate Cloud account.

请查看快速入门教程，或注册免费的 Weaviate Cloud 账户。